A feedforward artificial neural network is a category of machine learning models, which includes, as core concepts, directed acyclic graphs and connection weights. The neurons of a neural network may be partitioned into layers, such as input, hidden, and output layers. The neurons of an input layer do not have associated activation functions. An activation function associated with a neuron may be logistic sigmoid, hyperbolic tangent, linear, or identity.
A neural network must be trained before it can be used for prediction. Training entails determining the weights. Training usually involves solving an unconstrained optimization problem. After training, the network can be used for prediction by propagating the inputs and computing the values of the output neurons.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.